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Introductiont A Process Approach to Mental Ability 



This report, and the research project it describes, is based on 
an attempt to pose and explore alternatives to certain traditional con- 
cepts of mental ability. Conceptions of mental ability stem to a major 
degree from two traditions of mental testing? the psychometric and the 
clinical traditions. 

The psychometric approach generally describes the mental abilities 
and aptitudes of an individual in terms of how the individual solves a 
set of problems assembled into a test. The descriptions provided by 
tests emphasize one feature of the individual's solution: the correct- 
ness of the answer he finally offers as ”his” solution. Scores, and 
thus the individuals obtaining the scores, may be interpreted by reference 
to a population norm, by predicted performance on a criterion, or by 
reference to a hypothetical trait, usually defined as an underlying 
dimension that can summarize and account for covariation of the scores. 
All of these Interpretations assume a relationship between amount of 
ability and number of problems solved correctly; they generally do not 
carry information about the processes by means of which the individual 
arrived at his answer. 

Lack of information about process is a charge leveled by clinical 
testing advocates on the psychometric approach. The clinical approach 
substitutes for the impersonal paper-and-pencil format of group tests, 
a situation in which the subject is observed by a skilled observer while 
solving problems. It supplements the objective score with the impressions 

It derives its interpretations less from the results of 



of the observer. 



statistical analysis and cop?parison with other individuals, and more from 
the subjective interpretations of the observer. While the clinical 
approach is more often identified with the study of emotional and 
personality characteristics than with the study of mental ability, it 
is \ised extensively in Individual intelligence testing and in the 
diagnosis of brain damage (Phelps, 1964). Because of the clinical 
tradition* s emphasis upon subjective appraisal and wholistlc interpre- 
tations, this approach has developed neither techniques nor concepts 
for discussing process variables which are comparable to the elaborate 
correlational methodology and factor theory of the psychometric approach. 
Thus, while a clincian*s analysis of a single individual may present 
detailed discussion of his problem solving processes, recorded descrip- 
tions and comparison of individuals are likely to be in terms of scores 
derived and interpreted by psychometric principles. 

The psychometric approach, with its emphasis on scores and traits 
has not generated study of Individual differences in process, while the 
clinical approach has not developed the conceptual and methodological 
tools. Studies of individual differences in process would have both 
theoretical and practical value. An understanding of human thou gh t 
processes will have to take account of similarities and difference 
among persons^ and attempts to develop improved educational procedures 
will require knowledge of how thought processes affect ay*d are affected ■ 
by learning, and concepts of human differences that suggest ways of 
modifying them. Moreover, a number of recent theoretical and technical 
developments can contribute to a program of research on the topic. In 
the next few pages, we sketch briefly some lines of research that 



contribute conceptually to the differential study of thought processes, 
then outline some potential applications of such work, some thoughts on 
its impact on theory, and some suggestions concerning the research 
needed to achieve practical and theoretical Impact. 

Relevant Research 

Research developments relevant to a process approach to individual 
differences have occurred within the factor analytic tradition itself. 

In the study of cognitive development and in studies of human information 
processing. 

The two major concepts of human intelligence- global ability and 
component factors -have both emphasized traits that differentiate 
persons and that are evidenced by difference in products of problem 
solving. H\mt (I96I) has suggested that intelligence can profitably be 
conceptualized in terms of processes, citing concepts used in develop- 
mental studies by Piaget and in computer models. Other authors have 
assumed that a process approach to intelligence requires heretofore 
undeveloped knowledge of physiological functioning (Tuddenham, 1962). 

Modem factor theories provide a beginning to the problem of 
bridging the gap between concepts of trait and of process. Guilford’s 
(1967) structure of intellect model defines factors in terms of the 
intellectual operations to be performed, the contents upon which they 
are performed and the product that results. Each ability, therefore, is 
defined in terms of a -specific type of problem. The construction of a 
factorlally pure test requires more and more carefully refined problems 
and becomes more and more abstract as compared to real-life problems. 
Guilford has discussed the relevance of factors in the structure of 
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intellect to solving complex problems, learning and creativity, suggest- 
ing that complex problems will vary in the abilities they require, and 
that people will vary in the configuration of abilities they bring to 
bear on a problem and the sequence in which they call upon these abilities* 
Perhaps his most explicit statement of a possible relationship between 
trait and process employs the concept of function; 

”If each person has a characteristic level of ability 
to perform in each of [the] respects [represented by the 
cells in the structure of intellect model], he is certainly 

N. 

performing in these* • .ways* In other words, we may say that 
he has functions of [each of] these ***types* The primary 
intellectual abilities may therefore be conceived as ways 
of functioning within individuals as well as ways in which 
individuals differ from one another*" (Guilford, 1961) 

Analysis of factors as "ways of functioning" suggest questions 
at two levels of analysis: (a) What is the nature of the functions 
involved? Do individuals differ in the processes by means of which 
they solve factorized problems, as well as in the number of problems 
they can solve? (b) How do these ways of functioning operate and 
interact in the solution of complex problems? These questions are 
consistent with Guilford *s conception of factors, and do not conflict 
with the empirical evidence defining factors* They do, however, call 
for concepts and techniques different from those of trait theory and 
factor analysis* 









Studies of cognitive growth hy both Piaget (see Flavell, 1963) and 
Bruner (Bruner , et al. , 1966) emphasize variations in ways of functioning, 
both as a way of differentiating children at different ages or growth 
stages, and of comparing the developmental status of children. While 
the concepts employed by Piaget and Briiner differ in several important 
respects 9 both consider cognitive functioning in terms of the encoding 
and transformation of information. The resulting conception of intel- 
ligence differs in important ways from that of factor theozy. As Hunt 
(19hl) points out it portrays intelligence as developing more flexibly 
and as more amenable to educational modification. It is also more 
likely to describe an individual in terms of the sequence of cognitive 
events characterizing his solution of a problem, rather than In terms 
of a collection of trait scores portraying his ability to solve collections 
of different types of problems. 

The approach which has In recent years made the most persistent 
and ambitious attack on the problem of identifying though processes has 
been information processing psychology. This c^jproach derived much of 
its impetus from a source outside traditional psychology— the program- 
ming of computers to solve complex problems. Examining the programming 
techniques required to accomplish such '»artificlal intelligence," some 
psychologists postulated that programs might serve not only to supplement 
human cognitive processes but also to provide theoretical models of them. 
Thus a new technique, computer simulation, came into being. Computer 
models provided a method for detailed and rigorous specification of 
postulated cognitive processes, and a built-in "sufficiency test" of the 



specifications* The mechanics of programming facilitated the statement 
of cognitive processes in terms of the vicissitudes of information in a 
processing system - its coding, storage, retrieval, and transformation* 
This conception found kinship with certain ideas within Gestalt psycho- 
logy, notably those of Wertheimer (1945) and Duncker (1945)# -nd with the 
European systems summarized hy de Groot (1966)* Prom the combination 
has developed a lively new information-processing approach to psychology 
(Feigenbaum and Feldman, 1963; Bruner, Goodnow, and Austin, 1956)* 

Unlike many clinical and Gestalt approaches to process, this approach is 
elementistic rather than holistic, and is insistent upon discipline and 
detail in theory statement* However, it accepts the validity of complex 
internal events, and eschews the assumption that such processes obey 
the laws of classical and operant conditioning* 

Its emphasis on process has given information processing psychology 
two characteristic methodological features t detailed protocal analysis 
as a means of data collection and analysis, and simulation model building 
as a means of stating and studying theoretical processes* Problems of 
measurement are discussed below* Here we may summarize some properties 
of models especiaUr relevant to the study of Individual differences* 
Information-processing models have several characteristic postulates, 
of which the following are especially relevant to the problem of defining 
variations in process* 

(a) Immediate memory is limited to a small number of units* 

(b) Long-range memory has fewer limitations, but its availability 
depends upon how it is organized for retrieyal* 

(c) Problem solving consists of searching the very large array 
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of possible solutions to discover a ••real" solution, i.e., 
one that conforms to the rules and that satisfies the 
demands of the problem* 

(d) The search is selective rather than exhaustive. 

(e) The search is accomplished bjj utilizing elementary 
information processes, e.g., comparing, finding 
differences and transforming, that are arranged into 
processes and sub->processes* 

(f) The selections are made on the basis of heuristics, 

or decision criteria designed to discover an acceptable 
solution rather than a best or optimal solution* 

(g) The selection heiiristics are organized hierarchically, 
e.g*, the result of a test determines a subset of tests 
from which the next selection is made* 

Models having these characteristics have been developed for a variety 
of problem types, including theorem proving (Newell, Shaw & Simon, 1958), 
forming symbol sequence concepts (Simon & Kotovsky, 1963) and serial 
concept formation (Gregg, 1967)* These models have been attempts to 
account for general problem-solving processes but they specify a set of 
properties with respect to which individuals may vary. Simon & Kotovsliy 
have compared the performance of several variants of a basic model, 
and suggest that the variants might serve as models of individuals* 

^y developing a model whose operation is subject to variation according 
in input parameters, one may develop a theory of individual differences 
in problem-solving process* Such a strategy can contribute both to the 
understanding of Individual differences themselves and also to the testing 



7 















of process models* 

Simulation models have from the beginning been constructed with 
an eye to Individual differences* Many can be "initialized'* in varying 
. ways or at varying stages Ir* the problem solving process (Gregg, 1966) , 
and comparison of performance of computer programs with that of persons 
has suggested classifications of persons according to the processes 
they use (Paige & Simon, 1966) * Study of the relationship between 
concepts of cognitive functioning suggested by models and those suggested 
by factor analysis has been proposed (deGroot, 1966, Green, 1964) • 

Green has posed the question of whether one can develop a collection of 
programs that could "take" a series of tests and reproduce the correla- 
tions (and hence factor structure) obtained empirically* He reviewed 
a number of existing models as an initial Inventory for such a collection* 
For the most part, however, modelers have sought to build a general, 
rather than a differential psychology of thou^t* 

Applications of a Process Approach 

The applicability of ability tests has been a major factor in 
their continued study and in the form in which they evolved* They are 
highly effective for predicting success in tasks requiring native or 
already well-developed intellectual skills, emd as a consequence, have 
been widely used in education and industry* Procedures for item 
selection and scoring are designed to maximize such predictive value, 
and have resulted in stable tests, relatively insensitive to extraneous 
influences* It seems unlikely that process oriented measurement will 
improve substantially on the merits of traditional abilities tests for 
predicting such criteria* However, the same qualities that make tests 
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useful for prediction of quantiative criteria my make them ineffective 
for other applied purposes* By the same token, there are several kinds 
of applications for which process oriented descriptions would seem 
particularly useful. 

For identifying deficiencies in problem solving, process descrip- 
tions would have the merit of revealing the specific nature of the 
defect rather than its mere existence. Bloom and Broder (1950) found 
differences in the way college students approach the solving of exami- 
nation-type problems which produced wide variations in success, but 
which were not entirely attributable to the background or knowledge of 
the students. These differences suggested specific remedial procedures 
for groups of students. Well developed process descriptions could be 
valuable for designing educational programs for students at all levels 
of ability, possibly including the retarded as well as the normal and 
superior. Such an approach is particularly needed for developing 
techniques of adaptive education which would prescribe for the indivi- 
dual student the particular sequence of instruction host suited to his 
own needs (Cronbach, 1966), 

Process descriptions would also be useful ir. curriculimi evaluation. 
Standard achievement tests, designed to assess stable individual ability 
and attainment, have proven in many studies of curricula to be insensitive 
to curriculum variation, and, more to the point, not reflective of the 
objectives set down by the curriculum developers. The objectives of 
innovative educational programs have proven difficult to define in terms 
of test items. Part of the problem is that the objectives refer to complex 
cognitive processes, while test items, in general, try to capture behavior 
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at a given point in time. The problem is particularly acute when 
objectives are explicitly conceived in process terms, such as mode of 
Inquiry and multi-level understanding of a work of literature (Forehand, 

1966, b, c, d, Hastings, 1966), These problems call for new approaches, 
both methodological and theoretical, to the conceptualization of 
aptitude and achievement (Cronbach, 1963, 1966, Forehand, d). 

Other possible applications include the assessment of high level 
intellecttial aptitudes, such as scientific creativity, which are poorly 
predicted by traditional tests and lend themselves to description in 
terms sequential problem solving processes. 

These practical values are yet to be accomplished. Their realization 
will require a great deal of research leading to the conceptualization 
of individual differences in processes, development of techniques for \ 

assessing such differences, and establishing the usefulness of the | 

I 

techniques. Some suggested directions for such research and development I 

i 

are outlined below, I 
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Problems for Research 

Characteristic of the process approach is a concern for the sequence 
of intellectual operations that intervene between the presentation of a 
problem and the bebavior that identifies the subject* s solution. This 
concern suggests three emphases for research: 

(1) The design of procedures to gain evidence of the 
intervening processes, to externalize the internal 
events that are of interest, 

(2) The definition of measures to characterize a subject's 
processes, based upon the unorganized data thus obtained, 
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(3) The development of* concepts relating process measures 
to other variables* 

The ‘ extemalizatlon problem * There have been a number of attempts 
to develop process measures within the context of mental testing* Most 
methods have used one or both of two techniques for the externalization 

d 

problem: the use of problems that maximize the necessity for overt 

response during problem solving^ and the control of information relevant 
to solution* Both techniques are illustrated by tab item techniques 
used in studies of trouble shooting and diagnosis* The problem is solved 
by testing sequential liypotheses^ pulling tabs to determine thB results 
of tests or to obtain further information, and thereby leaving a record 
of successive steps in the solution* Tab- item tests have been designed 
in both linear (Glaser, et al*, 1954; Rlmoldi, et al, I 96 I) and branch- 
ing (McGuire, 1963) versions* In the branching form, the set of alterna- 
tives available to the examinee depends upon the outcome of previous 
decisions* More elaborate procedures can be Implemented with the aid 
of a computer using many of the techniques applicable to computer-aided 
instruction (Swets & Peuerzeig, I 965 )* 

Information control and forced overt response are also central to 
the problem solving and Information (PSI) device (John, 1957; John & 
Miller, 1957; Blatt & Stein, 1959; Blatt, 1961)* This apparatus 
presents a programmed logical problem; the subject must discover the 
logical structure of the problem and then solve it by means of queries 
put to the machine* The concept formation problems employed by Bruner, 
Goodnow and Austin (1956) have similar properties* 
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The externalization problem has also been a recent concern of 
information processing psychologists. The classic method for studying 
human information processing has been the analysis of **think aloud** 
protocols. Recent developments for the study of protocols Include 
procedures for detailed graphic portrayal of protocols (Newell, 1966) and 
supplementation of protocols ty aye-movement recordings (Winikoff, 1967), 
Other techniques, used to provide evidence of processing include the 
development of special problems for which alternative hypothesized 
processes would give different answers (Paige 6 Simon, 1966) and use 
of problems in which sequential responses on the way to the solution 
are required (Hayes, 1966), 

In general, an approach to individual differences in problem- 
solving process would contrast with traditional test methods on the 
following pointer 

1, It would Involve complex problems, requiring a sequence 
of decisions, in contrast to the short and purportedly 
unidimensional problems usually presented in an ability 
test (Forehand, 1966a, Taylor, 1966), 

2, It would permit the subjects to be in a range of states 
at a given time conditional upon prior strategies 
adopted and information obtained, 

3, It would provide a record of steps taken in moving toward 
a solution. 

One paradigm for such a technique would be a computer-aided testing 
program, A student seated at a console would be presented with a 
problem and a pre-determined but incomplete set of information relevant 
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to the solution. The student may ask questions and receive answers from 
a hank of Information. At points determined hy a student's progress , he 
might be asked to state his present analysis of the problem and his 
current strategy. The technical problems of operationalizing such a 
procedure are difficult but they are not Insoluble and technical 
Implementation Is not required for the paradigm to be useful. Conceptual* 
problems, however, need to be investigated prior to its implemntation. 
These problems Include the development of realistic problems; observa- 
tion and classification of ways in which subjects analyze problems, seek 
information and pursue strategy; and analysis of the meaningfulness of 
these classifications. Research on these problems ml^t well begin with 
broad, poorly defined problems, conducted by actual interaction between 
experimenter and subject, and proceed to the development of standard 
problems, principles for responding to subjects in standard ways, and 
objective recording of subjects* responses* 

This paradigm is intended to be a guide to thinking about the 
problem of process measurement rather than a necessary end result of 
research. Barts of the kind of research proposed above can be accomplished 
with simpler experimental sitmtlons. 

The description problem . Once a procedure for externalizing process 
has been devised, the investigator is confronted with the task of organiz- 
ing the data thus obtained to provide useful and communicable descriptions 
of the individual. The data will generally not be in the form of scores, 
and attempts to combine them into scores creates the danger of concealing 
the process features originally sought. Two ways of constructing 
descriptions, one empirical and the other theoretical in emphasis, may 
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be suggested* 

Process data lends itselT to the derinitlon of a variety of 
descriptors for example, the amount and qtiality of information sought, 
the nature of hypotheses offered at various stages, and the nature of 
the strategies stated. Such descriptors might be "tagged" to indicate 
the time or stage in problem solving at which they occur* An appropriate 
way of analyzing such descriptors would be pattern or configural analysis 
(McQultty, i960)* Such a procedure would indicate the frequencies of 
various patterns occurring, identify groups of subjects whose processes 
are in some sense alike, and permit study of empirical properties of 
persons having particular patterns* 

The second approach is to express descriptions of individuals In 
terms of information processing models* A process approach to individual 
differences will, in the long run, require a means for describing in 
detail the cognitive processes of an individual, contrasting them with 
those of other individuals, and generating implications that go b^ond 
the observation of a particular protocol* Let us assume for the moment 
that basic processes are properties of general human functioning, and 
that individual differences result from variations in the use and 
patterning of these processes* An explicit theory derived from this 
assumption would postulate [a] a sufficient set of processes for the 
solution of a class of problems, and [b] a set of mechanisms governing 
the sequential operation of these processes, variations in which would 
mirror variations among subjects* Information processing models are 
designed to meet the first of these goals (Newell & Simon, 1963) * Tie 
second calls for models which incorporate parameters, variations of which 
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can generate a population of hypothetical individuals and a detailed 
tracing of the processing of any particular member of the population. 

In the long run, information processing models of one kind or another 
will be the language of a process approach to individual differences, 
much as factor analysis is the language of a trait approach. Such 
models might take a number of forms: they may, for example, consist 
of equations postulating functional relationships, lavs governing 
conditional probabilities, or logical arguments. However, the technique 
which now seems most appropriate for developing and stating such models 
is computer simulation. Some work on the use of simulation for this 
purpose has been done by Joyner (1966). Among the methodological 
problems Implied by this strategy are the development of ways to write 
models, which, with appropriate instructions, will produce a set of 
varying solutions to the same problem, and of ways of matching variations 
with the behavior of persons. 

The development of a process approach to ability measurement will 
require a great deal of experimentation with measurement techniques, 
accumulation of results concerning processes used by individuals, 
construction of concepts aroxmd which to organize results and theory. 

The project reported here was intended primarily to explore techniques 
for defining variations in process operationally, and secondarily to 
gather some data on characteristics of process-oriented measures. During 
the course of the project, we have tried many kinds of techniques. 

Two of them have reachec the stage of development that we describe 
their rationale, examine some empirical features, and suggest roles 
that they may play in research and application* One of the approaches 
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is heavily empirical in emphasis, the other based on hypotheses about 
subjects* processing mechanisms. 

In section 2 of this report, we describe a study of students' 
attempts to solve a complex, realistic problem. The original data are 
open-ended responses to questions posed at various stages of problem 
solving, and individuals cure described in terms of empirical response 
patterns. The study reported in section 3 was based upon more explicit 
hypotheses about processes that occur during solution. This study 
focuses on a narrower range of problem solving, uses a data gathering 
procedure designed to introduce greater observational control., and 
describes individuals in terms of specific hypothetical process variables. 
In each of these sections, we shall consider the potential value of the 
techniques for research and application, and suggest extensions of them. 
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Analysis, Strategy and Information Seeking in the Solving 

of Complex Problems 

In Section I of this report, we outlined several characteristics 
of a data gathering procedure for obtaining information about problem 
solving process: use of complex problems, provision for a rsmgs of 

descriptive states, and provision of a record of steps taken subjects^ 
We proposed that such a procedure present to the subject a problem and 
a predetermined but incomplete set of Information relevant to solution, 
allow the subject to ask questions and receive answers from a bank of 
information, and require the subject at specific points, to state his 
present analysis of the problem and his current strategy. 

We have studied problem solving by college students in a situation 
having the above properties. In contrast to many problem-solving studies, 
we have used open-ended procedures: a realistic problem without a clearly 
best solution, interaction between the subject and the experimenter, and 
self«composed verbal responses by the subject. The aim of this strategy 
is to discover features of Individual variation in solving such complex 
problems which would bo concealed from view by more structured procedures. 
Its cost is an inevitable degree of ambiguity in the results. We hope 
to develop hypotheses by such a procedure which may then be tested under 
more carefully controlled conditions. 
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The subjects of this stud 7 , who are students in a teacher training 
program are confronted with a problem similar to professional problems 
encountered by high school teachers (see Appendix A). It Involves a 
student with a learning problem, which, realistically, is only partially 
described in the presentation. To make a useful attempt at solving the 
problem, a subject would need to know more about the student, the problem, 
and thQ source of information. The collection of data involves three 
stages. After the problem is presented (Stage I), the subject provides 
[a] his present analysis of the problem; [b] a list of additional infor- 
mation that he requests for further consideration together with the source 
from which he would seek the information; and [c] a statement of his 
current strategy for solving the problem. At Stage II each subject 
receives feedback in response to the particular set of questions ho 
asked, and he responds in the same way as at Stage I, Feedback is 
developed from a standard model of the problem, and subjects who ask 
the same question get the same answer. Stage III repeats this process. 
Obviously the process could go on indefinitely; we stop at three stages 
in the present study for convenience, A copy of the problem and the 
response form are presented in Appendix A, 

A scoring system has been developed which describes an Individual 
by means of the following variables. We will use the symbols presented 
here throughout the discussion, A scoring guide is presented in the 
Appendix B, 

(1) Analysis ; The subject’s present analysis is characterized by 
whether the subject [a] zeroes in on a single hypothesis concerning the 
bases of the problem (S) or [b] considers multiple alternatiyes (M), 
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(2) Infonnation sought ; The number of items of Information 
requested, provided that they are appropriate as the problem is stated, 
is counted. A scoring guide provides means of resolving ambiguities 
and differences in the way questions are worded. For the analyses 
presented here, each subject was classified according to whether the 
amount of infonnation sought was high (H) or low (L) relative to other 
subjects at the same stage of problem solving. 

(3) Steategy; Three types of strategy can be identified, [a] A 
conditional branching strategy contains an explicit or Implicit if-then 
statement; the stated strategy is contingent upon results of inquiries 
that have been made, [b] An open alternative strategy leaves alternative 
courses of action open (including obtaining more information) but does 
not contain a specific linkage between information sought and courses of 
action. Finally, [c] an unidirectional strategy states a course of action 
without reference to results expected from information seeking. Since 
relatively few of our subjects adopted a conditional branching strategy, 
we have combined this category and the open alternative category. We 
shall term the combined category the open alternative strategy (O), as 
contrasted with the unidirectional strategy (U) . 

Since data susceptible to analysis in these terms is obtained at 
three different times, each subject's response pattern can be described 
by means of nine characteristics, which we shall term descriptors . 

Analyses to date have explored properties of descriptors and patterns of 
descriptors, and their potential usefulness for characterizing a subject's 
problem solving processes. 
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Some Properties of Descriptors 

We occasionally use to characterize descriptors and patterns three 
outside measures of ability and achivement. Scores on the SAT verbal 
examination; scores on the SAT quantitative examination, and cumulative 
grade point average^. We are not primarily concerned, however, with the 
predictive merits of our descriptors* An attempt to predict cumulative 
grades would be especially unwise, because, as we shall see, our sample 
includes students with a wide variety of academic programs, and criteria 
for grades vary in the different programs* For example approximately 
one-third of our sample consists of students in fine arts (music, painting 
and design) , whose performance in many of their classes rests more on 
artistic talent than on academic ability. 

Table 2-1 shows relationships between the nine descriptors and 
three external variables. There is little evidence that the descriptors 
individually are related in a systematic way to these measures* (We have 
observed some indications that there may be some systematic relationships 
within groups, e.g. , fine arts or liberal arts and science students; we 
do not as yet, however, have sufficient data to substantiate these 
relationships) * Relationships among the descriptors are summarized in 
Table 2-2. The higher relationships in the table seem to make sense. 

1 

The University uses a 4-polnt system with A=4, B=3, C=2, D=l, Failure=0. 
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Mean Test Scores and Grades for Subjects Described by 

Process Descriptions 



(Table 2-1) 
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. Analysis: A=Multiple, B-Single 

. Information: A=High, B=Low 

. Strategy: A=Open- alternative, B=Unidirectional 

Significant at .05 level 




Table 2-2 

Intercorrelations Among Descriptors 
(Phi Coefficients) 



Descriptors i 


2 3 £ 5 6 


7 


8 


£ 


1. Analysis, Stage I 


.18 .25* .16 .08 .09 


-.12 


-.24 


.33* 


2. Information, Stage I 


.12 .13 .21 .04 


-.13 


-.06 


.08 


3. Strategy, Stage I 


.33* .28* .17 


.16 


-.06 


.16 


4. Analysis, Stage II 


.35* .24 


.35* 


.04 


.17 


5. Information, Stage II 


.06 


-.07 


.06 


.09 


6. Strategy, Stage II 




.06 


.16 


.19 


7. Analysis, Stage III 






.23 


-.05 


8. Information, Stage III 








-.11 


9. Strategy, Stage III 











* Significant at .05 level 



For example, subjects who adopt an open-alternative strategy at the end 
of Stage I (variable 2) are likely to have considered multiple hypotheses 
concerning the problem (variable 1) , maintain such and analysis at Stage 
2 (variable 4) and seek a relatively high amount of information at that 
stage (variable 5) • It will be noted that for the most part, a parti- 
cular type of descriptor (for example, information seeking) is not 
related in a systematic way to similarly defined descriptors at other 
stages (cf. variables 2,5,8). Such a result is illuminated by examining 
cross-stage patterning of descriptors, as is done below. These analyses 
suggest that such descriptors change systematically from stage to stage, 
with different types of subjects having different patterns of change. 

Thus there is no simple pairwise relationship among these descriptors. 

Patterns of Descriptors 

Since the three descriptors at each stage can each have two values, 
there are eight possible patterns of the three predictors at each stage. 

lists these patterns^ shows the frequency of their occurence at 
each stage and presents the mean scores on the three external measures for 
persons obtaining each pattern. 

It will be noted that the popularity of certain patterns varies with 
the stage of problem solving. This is particularly evident with respect 
to the analysis variable; the proportion of persons who consider multiple 
alternatives decreases from .61 at Stage I to .35 at Stage II to .24 at 
Stage III. In general the shift seems to be one of narrowing from an 
open, exploratory approach, to a focused approach. For example, the 
proportion of subjects who combine multiple-hypothesis analysis and open- 
alternative strategies changes from .33 to .26 to .07, while the proportion 



23 






(Table 2 




who present single-alternative analyses with xmidirectional strategies 
increases from ,30 to ,40 to ,41, 

In only one of the columns of Table 2-3 are the mean scores 
indicated to be significantly different from one another by analysis of 
variance (See Table 2-4). This is the grade column at the first stage. 
The pattern of the shifts in mean scores of this and of the other 
columns suggest that higher scoring students in particular are making 
the shift from tho open to the focused problem-solving strategies. In 
general y mean scores for subjects having M— analysis and 0— strategy 
descriptors decrease from Stage I to Stage III, while those for subjects 
having S and U descriptors increase, 

Cross-Stage Patterns 

A concern with process Implies Interest in patterns of problem 
solving across stages, and such an interest has been indicated in our 
discussion of shifts in patterns from stage to stage. Since there are 
9 dichotomous descriptors, there can be defined 2^ possible patterns of 
all descriptors. Since we have many too few subjects to allow for the 
occurrence of all of these patterns, we concentrate on [a] cross-stage 
patterns of a given type of descriptor (i,e,, analysis, information and 
strategy), and [b] certain a-priori patterns that have special relevance. 

Figures 1, 2 and 3 present cross-stage patterns of descriptors on 
the analysis, information and strategy variables respectively. They are 
arranged to show groupings of subjects at each stage, and how these group- 
ings break up at the next stage. With respect to each group of subjects, 
we show [a] the number attaining the pattern [b] mean scores on the SAT-V 
and SAT-Q of these subjects and their mean grade point average, and [c] 



Table 2-4 (page 1 of 2) 
Analyses of Variance of Patterns within Stages 



A. Stage I, SAT-V 





SS 


df 


MS 


F 


Total 


551,906 


60 






Within 


69,127 


7 


9,875 


1.08 


Between 


482,779 


53 


9,109 




B. Stage I 


, SAT-M 










SS 


df 


MS 


F 


Total 


775,876 


60 






Within 


78,907 


7 


11,272 


.86 


Between 


696,969 


53 


13,150 




C . Stage I 


, GPA 










SS 


df 


MS 


F 


Total 


24.1041 


60 






Within 


8.2720 


7 


1.1817 


3.86 


Between 


15.8321 


53 


.2987 




D. Stage II, SAT-V 










SS 


df 


MS 


F 


Total 


510,655 


57 






Within 


109,966 


7 


15,709 


1.96 


Between 


400,689 


50 


8,013 




E. Stage 11 


:, SAT-M 










SS 


df 


MS 


F 


Total 


741,973 


57 






Within 


136,889 


7 


19,555 


1.62 


Between 


605,084 


50 


12,101 




F. Stage II 


, GPA 










SS 


df 


MS 


F 


Total 


15.9391 


57 






Within 


2.5133 


7 


.3490 


1.30 


Between 


13.4258 


50 


.2685 





Table 2-4 (page 2 of 2) 



Analyses of Variance of Patterns within Stages 



G. Stage III, SAT-V 





SS 


df 


MS 


F 


Total 


361,990 


41 






Within 


41,791 


6 


6,965 


.76 


Between 


320,199 


35 


9,148 




H. Stage III, SAT-M 










SS 


df 


MS 


F 


Total 


456,116 


41 






Within 


38,882 


6 


6,480 


.54 


Between 


417,234 


35 


11,921 





I. Stage III, GPA 



Total 


12.4283 


41 




Within 


2.1078 


6 


.3513 1.19 


Between 


10.3205 


35 


.2948 
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the percent of students majoring in fine arts as compared to those in 
liberal arts and sciences. 

The comparison between fine-arts and other subjects was instituted 
when casual observation suggested that there were certain patterns 
likely to be obtained predominantly by fine arts students and others 
predominantly by liberal arts or science students. Differences in 
cognitive processes among these students would not be surprising^ they, 
at least at this university, are selected by different criteria, taught 
with different emphases, and graded on different achievements. Students 
in fine arts do a large proportion of their work in the practice of their 
art, and are graded to a large degree on artistic skill and creativity. 

As a consequence their grades are less readily predictable from academic 
predictors than are those of the students in more academic programs. 

Since the present subjects are teacher— trainees, their programs have 
more academic courses than do other students in fine arts, but compared 
to liberal arts and science students their programs still have a heavier 
emphasis on artistic practice. 

Comparison of these tables will be facilitated if we can assume 
some similarities of function among the descriptors. Some of the performances 
can be described a s_scanning the environment and others as focusing on a 
solution or narrow range of solutions. Let us postulate that formulating 
multiple-hypothesis analyses, high information seeking and posing open- 
alternative strategies all represent scanning behavior (S) while perform- 
ances producing single-hypothesis analyses, low information seeking and 
unidirectional strategy represent focusing (F) . Using these concepts. 
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mj note the following features of Figures 1-3 » 

1« The patterns are not equally likely to occur* Most 
subjects either begin by scanning and focus on Stage 
2 or 3 or maintain the type of problem solving they 
begin with (i.e*, consistently scan or consistently 
focus)* Comparatively few subjects switch from 
focusing to scanning or revert to a previous type 
of response* These relative frequencies may be 
summarized as, follows: 

Variable 



Description of Pattern 


Analysis 


Information 


Strat«g 7 


a* 


Scan, then focus 


5ose 


30f 


3W 


b* 


Maintain initial response 


30H 


yzk 


Wt 


0 * 


Focus, then scan 


12!( 




13!t 


d* 


Revert to previous response 


85f 


ya 


1758 



2* Certain groups of students appear more likely to adopt certain 
patterns of response* . For example the patterns that represent 
initial scanning, followed by focusing at Stages 2 and 3 
characterize a disproportionately large proportion of fine 
arts sttidents* Fine arts students form a substantial majority 
of subjects displaying this pattern on the analysis and 
information variable, although the percent of art students 
in the total sample is approximately 30* Similarly, students 
who have high scores on the aptitude tests are represented 
disproportionately often in the patterns reflecting initial 
scanning and focusing on both Stages 2 and 3« 
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To obtain cross-stage descriptions which take in to accoimt all 
three variables ^ each intra-stage pattern was classified as representing 
scanning or focusing^ on the basis of the elements within the pattern* 

If two or three of the three descriptors in a pattern were in the 
scanning direction^ the pattern as a whole was termed scanning* Otherwise 
it was called focusing* The cross-stage patterns resulting from this 
rescoring are described in Figure 4* 

The conclusions drawn above are consistent with this unified 
analysis* The percentages falling in the categories discussed above are 



a* 


Scan^ then focus 


48^ 


b* 


Maintain initial response 


35if 


c* 


Focus 9 then scan 


10^ 


d* 


Revert to previous response 





The conclusions concerning patterns favored by art students and by hi§^ 
scoring students are also supported* In this case there are patterns 
of one and two subjects who score higher then the SSF group^ but the 
mean scores of subjects in this pattern are the highest anong the 
frequently occurring patterns* 

Discussion 

This study summarizes efforts to develop procedures to obtain^ 
score and analyze data concerning processes used by subjects in solving 
complex problems* The strategy of using sequential problems with 
information sou^t and received sequentially^ amd of eliciting data 
about the subjects' analysis and strategy at various stages of work 
appears to be a useful one* The empirical results gathered thus far 
suggest conclusions that are reasonable and that have important 
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implications for the study of individual difference. These conclusions, 
]jovever, must remain hypotheses until more extensive data can be brou^t 
to bear on them. It seems reasonable to propose that research continue 
to refine the techniques further, to test the hypotheses more rigoro\isly, 
and to use the techniques for additional purposes. Research now planned 
along these lines includes the following studies. 

1. Continuing data collection on the cross-stage patterns, to 
permit more rigorous statistical analysis. 

2. Standardization of the procedures, permitting us to tost 

subjects individually, giving immediate feedback to questions 

/ 

from a standard data pool. 

3. Study of the generality of descriptors across problems and 
subjects to determine the extent to which process variables 
depend on the specific problem being solved and upon the 
academic background of the students. 

4. Study of problem solving process as an educational objective, 

A persistent problem in curric ulum evaluation has been a difficulty 

in matching curriculian objectives with behavioral measures. Maiqr attempts 

by psychologists to define educational objectives operationally have met 

with skepticism from curricultim developers because they are more limited 

in scope and more sUtic than the objectives as educators state them 

(Forehand; 1966d). Masqr curriculum objectives are conceived as processes 

rather than products. For example, curriculum development projects In 

social studies cite **mode of inquiry** as an objective. Mode of inquiry 

refers to the selection, and interpretation of evidence, development of 

hypotheses and drawing of conclusions about social phenomena. Literature 

educators also describe the imderstandlng of a work of literature as a 
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process* ( Forehand » 1966c)* 

The sequential decision-making Instruments discussed here are 
more nearly in accord with the aim of assessing educational outcomes 
as processes than are traditional psychometric tests* Problem analysis 
exercises may be developed in conjunction with a set of curriculum 
objectives* These exercises may be applied in a course having those 
objectives, and evaluated as a measure of the objectives* Analyses 
might Include (a) comparison of pre- and post-course performance of 
students in the course, (b) comparison of performance of students 
in the course with that of students in a similar course not having 
those explicitly stated objectives, and (c) comparison of the' information 
derived from these procedures with that obtained from traditional tests 
of achievement* 
















Information Organization and Strateg7 as Variables 
in Problem Solving 



A process approach to mental ability assmes that individuals vary 
with respect to such properties as the vay they perceive the problem^ 
the vay they seek^ organize and retrieve relevant information and the 
strategies they pursue* Goals of the approach include: 

a* The development of abstract descriptions or models of 
individuals* processing systems which would permit ' 
reconstruction and prediction of the individuals* 
responses to a given collection of problems* 
b* Generalization from these models to discover their 
relevance to other problem solving situations* 
e* Exploration of the usefulness of the models for 

eicplaining and modifying the behavior of the subjects 
in proactical situations* 

These goals require information about particular processing 
mechanisms* Ths purpose of this study is to explore variations in 
certain hypothetical processing mechanisms pertaining to organization 
of information and solution strategy* 

Description of Problems 

The principal problems of the study have the following properties: 
a* Information necessary for solving problems is 
given prior to presentation of the problems , in 




non-organized Tashioii* The subject records 
this infonnatlon before he receives instructions 
to the problem itself* The information record- 
ing sheet is ruled in squares ^ but contains no 
other restraints on how the Information can be 
recorded* 

b* Two kinds of information are given: (1) relational 

restraints which, taken together, permit all 
problems to be solved deductively, and (2) hints, 
which suggest solutions to problems that may or 
may not conform to the logical demands of the 
problem* 

The entire set of problems and instructions is presented in 
Appendix C (entitled, the "Couples Test")* To facilitate later 
discussion, we describe them briefly here* 

The initial instructions ar^ s follows; 

This test contains several puzzles based on the romantic 
adventures of five young ladies: Ann, Betty, Cathy, Doris, and Ellen* 
Each of the girls has definite ideas about acceptable boyfriends* They 
accept dates according to the rules given on the cards that you have* 
Take a' few moments to read over the cards* 

Now make a note of the information given on the cards on the 
Information Sheet that you have* You will use this sheet to refer to 
while solving the puzzles* You may write down the ijifonoation in any 
order you wish* 

When you have finished, return the cards to the experimenter. 
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Ramember that each girl obeys her own rules consistently in the 
problems that Tollow* 

Do the problems "in your head". Do not make notes or write down 
anything except the answers you wish to give* 

Each of the following statements is given on an information card. 
1* Betty will not date chemists* 

2* Ellen will not date dentists* 

3* Ann prefers engineers* 

4* Doris will not date engineers* 

5» Ann will not date architects. 

6. Cathy will not date engineers* 

7* Betty prefers bartenders* 

8. Doris will not date dentists* 

9* Ellen will not date bartenders* 

10* Ellen prefers architects* 

11* Betty will not date dentists* 

12. Doris prefers chemists* 

13* Cathy will not date bartenders* 

14* Ellen will not date ch^ists* 

15* Ann will not date bairtenders* 

There are four sets of five problems each* The first set^ which is 
"^ypAcal of the format ^ is as follows t 

1* The five girls have recently been escorted by five young men? 
Al, Bert 9 Carl^ Don^ and Ed* One of the men is an architect^ 
one a bartender, one a chemist, one a dentist, and one an 
engineer. lour problem is to discover which is which* 
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Ann has dated Al. 

Ellen has dated Bert* 

Doris has dated Don* 

Betty has dated Al* 

Cathy has dated Ed* 

Betty has dated Carl* 

Write the profession of each man beside his name below* 

WA^ PROFESSION 

Al 

Bert 

Carl - 

Don 

Ed 

Subjects were allowed five minutes for recording the information^ 
and five minutes for each of the four sets of problems. 

In referring to these problems ^ we shall use initials to designate 
both the problems and the solutions* Thus the problems will be designated 
A (for Al), B (for Bert)**.T (for Tom), The solutions will be designated 
A (architect), B (bartender), C (chemist), D (dentist) and E (engineer). 
Thus Problem B will refer to the second problem above, while Solution B 
will refer to bartender as a response* 

The "couples" problems were the focus of this study* In order to 
stuoy variables apsociated with performance of subjects, we have used 
two additional measures of information organization and logical thinking: 
The Word Matrix Test and a Syllogism Test* The Word Matrix Test is 
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suggested GullTord and Hoepfner (1966) as a measure of the factor^ 
S«nantic Relations, described as ^the ability to see relations between 
ideas or meanings of words"* For example, the subject may be required 
to place the words listed below in the blc;iks of the matrix so that there 
are meaningful relations among the words in each row and column* 
dog, penguin, canary, horse, minnow, shark 

Ostrich ' 

__ — Mouse 

Trout 

The syllogisms test presents standard logical problems, consisting of 
premises and conclusions. The subject judges the logical validity of 
the conclusions. The ability measured might be desribed in Guilford’s 
terms as evaluation of semantic Implications, and a similar test is 
suggested by Guilford and Hoepfner (1966) as a measure of that factor* 

Hypothetical Process Variables 

Studies of this instrument hsive focused on two properties of the 
subjects* responses: the way in which they organize the infoiination 
presented, cmd their relative use of the logical information and the 
hints, as implied by their patterns of responses to the problems* In 
each case, the procedure has been (a) to identify groups of subjects 
having similar properties with respect to these variables, and (b) to 
study relationships between these classifications and performance on 
the Couples Test and on tests of logical reasoning and of semantic 
organization* 
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Organization of Information 

Examination of the items of information presented above will reveal 
that they can be reorganized In a number of ways to facilitate their use 
in problem solving* They could be indexed according to girls* names^ 
assembling all of the material pertinent to a n&me in one place* They 
could be similarly organized by profession* They types of information 
(exclusions and preferences) mi^it be tagged or recorded separately* 
These modes of organization might be used in various combinations* 
Probably the most efficient technique would be to arrange the infommtion 
in a matrix, with one dimension keyed by girls* names and ttie other by 
professions, and to tag the "preference** items either by a special 
mark or by placing them separately* 

The information sheet of each subject was examined to determine 
the mode of organization used by the subject* Subjects were classified 
as having used function as a classification if they recorded preference 
and exclusion information in different sections of the information sheet, 
or if they tagged one type of information with some clearly identifiable 
mark, so that the function of the item is apparent at a glance* 
Frequencies of '^nP^s of organization for 41 subjects are presented in 
Table 3«*1* These results show thatr 

a* Girls* names were the major data used for classifying 
information* Only two of the 41 subjects did not 
classify according to that variable* 
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Table 3-1 




i 



Numbers of Subjects Using Various Types of Organization 



Type of organization 

N P F 
N P 
N F 
P F 
N 
F 
P 

None 



Number of Subjects 

12 

0 

21 

C 

6 

0 

2 

0 
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Total 



41 



P - 



Abbreviations: N - organization by girls* names 

organization by profession 
organization by function of item (infor- 
mation about exclusion or information about 



F - 



preference) 
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b» Approximately half of the subjects classified the information 
hy girls* name and separated out (either spatially or hy a 
special mark) the preference items from the exclusion items. 

Almost yjft classified hy both indexes and also sepeurated out 
the types of infoncation and classified hy either names 
or professions without differentiating type of information. 

None of the subjects simply wrote down the information 
without some effort at sorting it out. 

Effects and Correlates of Type <gf Organisation . 

Since the only information available to the subject while he is 
working the problem is that which he himself has recorded^ his ability 
to work the problems should be influenced hy the effectiveness of his 
organization. To examine this relationship^ and additional correlates 
of the organization variable ^ the subjects were divided into three 
groups: those with the NPF i^ttern, idio used all three of the available 
principles of classification^ those with the NF pattern^ who used two of 
the principles 9 and those with either the N pattern or the P pattern^ 
who used only one principle. 

We would expect the group using three principles to be the most 
effective in solving the problem. Table 3-2 and the first part of 
Table 3-1 support this hypothesis. The mean scores of the groups are in the 
order predicted, and the means are significantly different from one 
another. 

Is mode of organization related to ability to classify concepts 
to logical ability? Thble 3-2 and parts B and C of Table 3-3 show the 
relative performance of the groups on the Word Matrices Test and the 

o 
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Table 3-2 



Test Performance by Subjects Classified by Type of Organization 



Humber of Organization 
principles used 


N 


Couples 

Test 


Word 

Matrices 


Syllogisms 


3 


12 


17.0 


75.8 


21.0 


2 


21 


12.7 


64.8 


18.4 


1 


8 


11.7 


68.6 


16.3 
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Table 3-3 



Analyses of Variance of Test Performance by Subjects Classified 

by Type of Organization 
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A. Couples Test 



Source of Variation 


Sum of 




Mean 


F 


F. 

95 


F. 

99 




Squares 




Square 








Total 


681 


40 










Between Means 


178 


2 


89.0 


8.40 


3.25 


5.21 


Within Groups 


403 


38 


10.6 








Word Matrices 














Source of Variation 


Sum of 


M 


Mean 


F 


F. 

95 


F. 

99 




Squares 




Square 








Total 


5,697 


40 










Between Means 


985 


2 


492.5 


3.97 


3.25 


5.21 


Within Groups 


4,712 


38 


124.0 








Syllogisms 














Source of Variation 


Sum of 


M 


Mean 


F 


F. 

95 


F. 

99 




Squares 




Square 








Total 


605 


40 










Between Means 


111 


2 


55.5 


4.69 


3.25 


5.21 


Within Groups 


494 


38 


13.0 
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Syllogisms Test* In each ease, the differences are significant at the 
•05 level* In the case of syllogisms, the order of performance is the 
same as that for the couples test, while in the case of Word Matrices, 
the one-principle group has a hi^er mean score than the two-principle 
group. 

Problem Solving Strategies * 

There are several options available to the subject for handling the 
"preference” information he has been given* He can ignore it, solving 
all problems deductively, on the basis of the exclusion information; all 
of the problems are soluble this procedure* We shall term this 
strategy the "exclusion" strategy (ES)* On the other hand, he can use 
the Information about preferences whenever it is available to give him 
his answer* (For example, if Betty prefers bartenders, and Betty has 
dated Vic, guess "bartender" for Vic) * This strategy would sometimes 
yield correct responses, and sometimes yield predictable errors* We may 
term this procedure the "preference "strategy (PS)?' Let us assume that an 
individual adopting a preference strategy uses exclusion data to resolve 
conflicts (i«e*, idien the same solution occurs twice in a single set of 
problems), but otherwise makes a preference guess* 

To illustrate these strategies, consider the first problem of 
set 2 (Problem P)* Individuals using the three strategies mig^t proceed 



1 . . 

There are other possible strategies of course, such as forming hypotheses 
on the basis of preference information, and testing them rigorously by 
exclusion information* We have concentrated on the two strategies 
described here since they yield distinguishable predicted response 
patterns* 



as follows t 



Exclusion Strategy 



Preference Strategy 
B F, C F 

B prefers bartenders 



Given 



B F*, C F 



Observe from B will not date 
Information Chonists or Dentists 
Sheet: 



C will not date 
Bartenders or Engineers 



C -> no preference 



Conclusion: Architect (the only Bartender 



profession not excluded 
by either B or C)» 



It is assumed that a problem not solved by the data at hand is left 
blank until after other problems are worked, after idiich it is returned to* 
These two strategies may be described in detail by writing out a 
routine or sequence of operations that, when performed, will provide 
response patterns which would be characteristic of persons employing 
those strategies* Routines to accomplish this are presented in the 
Appendix D* It will be noted that while these routines are written as 
sets of instructions in English, they employ principles characteristic of 
computer simulations, for example, hierarchical organization cmd 
recursion* Thou^ we have not attempted to translate them into computer 
programs they would appear readily susceptible to such treatment* This 
might reveal ’’bu^” and* ambiguities, and hence sharpen their precision* 

For the moment, however, we are concerned with studying the reasonable* 
ness of the assumptions that produced them, and their usefulness for 
describing individuals* 

2 * 

That is, **Betty has dated Fred"* 
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Thess routines are based on the asumptlons about huwn problem 
aolrlng processes described above. They were not constructed by 
simulation. In the sense that they attempt to describe observed processes. 
If their assunqitlons are correct, however, the result would be simulation. 
They nay thus be termed hypothetical simulations. 

In the next few sections, we shall consider these hypothetical 
strategies, and the assumptions that go Into them. We begin by 
some Implications of the preference vs. exclusion assumptions for 
Internal relationships in the test. We then discuss the problem of 
matching individual response patterns with models, and finally 

further characteristics of persons classified by the concepts involved 
in the nodelse 

Internal Checks > 

Both strategies lead to the same set of response patterns for the 
problems of PSrt 1 (A-E) and those of Part 4 (P-T) . They lead to 
different patterns for PUrt 2 (Problems P-J) and Part 3 (Problems K-0). 
Therefore, the It) problems of sets 2 and 3 were used to classify subjects. 
As a first check on the assumptions, individuals were classified on the 
basis of their responses In problems F-J and these classifications were 
used to test internal Iqrpotheses regarding other items. 

For items F-J, the correct answers are A,E,B,C and D, in that 
order. A person using the exclusion strategy accurately would be eiq)eeted 
to obtain this pattern. However, the correct solution to all subsequent 
problems depends on the correct solution of the first. If the first 
problem of the set is kssed, all subsequent proWLSms would also be missed 
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t7 a subject pursuing this strategy. Similarly the last three problems 
depend on solving the first two. Thus, If errors occur, they should 
occur in a pattern. It should not be possible to miss items F and G, 
and get items H and I correct. The predicted preference strategy 
pattern is D,A,B,C,E, 

Using these guides, each, subject was classified as to whether his 
. pattern best matched the exclusion or the preference strategy, (Further 
discussion of the matching process and the degree of correspondence with 
the patterns occurs below). Twenty-five of the subjects were classified 
as using the exclusion strategy, and 15 as using the preference strategy. 
One was not classifiable. 

If we are approximately correct in classifying the subjects, certain 
hypotheses regarding responses to other items in the test ou^t to 
hold. In the third set of items (items K-O), the preference strategy 
predicts particular errors for items K,N and 0, (The predicted pattern 
is E A D C B, the correct pattern, C A D B E ) , Thus we would expect 
PS subjects to miss these items more often than ES subjects. On the 
first set of problems (A-E) and fourth set (P-T), both models predict 
the correct pattern, as mentioned above. However, for exclusion- 
strategy subjects, items C,D,S and T, ou^t to be relatively difficult, 
since their solutions depend on solving previous items in the set. For 
preference-strategy subjects, they are no more difficult than other 
items. Therefore, we ml^t expect ES subjects to miss these items more 
frequently then PS subjects. 

Results pertaining to these hypotheses and to the classifications 
are presented in Table 3-4. The first three problems listed in the 
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Table 3-4 



Percent of Subjects Missing Certain Items 



2 

Item Percent Missing Item Difference ^ 

(P-E) 





Preference 

Subjects 


Exclusion 

Subjects 








CN «= 15) 


(N = 25) 






F 


87 


24 


63 


- 


6 


100 


32 


68 




J 


100 


48 


52 




K 


47 


16 


31 


4.41* 


N 


87 


32 


55 


11.21** 


0 


80 


44 


36 


4.96* 


D 


7 


52 


-45 


7.84** 


E 


29 


60 


-31 


3.83* 


S 


13 


12 


01 


<1.00 


T 


33 


16 


17 


1.60 



* Significant at .05 level 

** Significant at .01 level 
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tabid ware used In the olassificatidn scheme; they ere presented to 
Indicate properties of subjects so classified. Results pertaining to 
problems K, N and 0 refer to the expectation that PS subjects would miss 
these items more frequently than ES subjects. All three of these 
differences are in the predicted direction and are significant. On 
problems D^E^S and the £S subjects were expected to miss more often. 
This hypothesis is bohie out for two of the problems » and the difference 
is not significant for the other two. 

Results to this point indicate that preference strategy 
exciuslon strategy are promising concepts for describing 
and that individuals show some consistency in their pursuit of a strategy. 
We nay now describe attempts to classify subjects more formally. 

Matching Response Patterns to Models . 

In Appendix D are presented routines for solving the couples 
problems by the exclusion strategy and by the preference strategy. 

These models assume that a subject tries problems in the order that they 
are given, and leaves responses blank if thsy are not soluble at that 
point, returning to them after all other items are completed. The 
predicted response patterns for each of 40 subjects to problem sets 
2 and 3 were derived by applying the routines to the problems, using 
the information as the subject himself had recorded it. Thus if a 
subject had recorded data erroneously (as 7 of them did) the predicted 
patterns for him will usually vary from those of subjects using accurate 
information. 




For the items of set 2 (F-J) ^ the preference strategy predicts 
the pattern D A B C E (for subjects who recorded information items 
correctly)* Of these the third and fourth answers are correct* A 
subject's pattern was termed a match to the model if he gave this pattern 
or if he blanked the items he was predicted to miss and got the ones 
correct that he was expected to get correct (i*e*, the pattern B C J* 
The predicted preference pattern for problem set 3 (item K-O) is 
E A D C By of which the second and third are correct* Thusy this 
pattern or the pattern ^ A D ^ ^ were termed matches with the preference 
strategy for this item set* Appropriate modifications of these rules 
were used when a subjects erroneously recorded information led to a 
different set of predictions for him* 

The exclusion stmtegy, followed as outlinedy will solve all 
problems correctly (again assuming accurate information recording)* 
Howevery because of the sequential nature of the problem if a subject 
makes an error in an early itemy there would be certain later items 
which he could not answer correctly by following this strategy* The 
procedure for predicting a subjects* pattern by this strategy was as 
follows: Whenever a subject missed an itemy his answer was substituted 

for the model's cuiswery and subsequent predictions based on responses 
thus recorded* Errors were not "counted against" a match with the 
model* Assuming that Information was recorded correctlyy the following 
patterns match the exclusion strategy pattern by these rules* Tte 
symbol " represents an incorrect response* 
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Problem set 2 



Problem set 2 



A S B .0 D 



C A D B E 



A E B C * 



C A » B E 



A E B 



C A « B * 



A E * « * 



C A * ♦ « 



n a a It 



C * * * » 



* ^ it n n 



a a a it 



n n a it n 



Thus the patterns termed matches with the ES and PS models are 
mutually exclusive* a pattern cannot match both* Since the criteria 
for defining matches differ under the two models, we do not attempt here 
to determine closeness of obtained to predicted patterns when the obtained 
pattern is not a match. Thus a given response pattern on a set of five 
items may match the ES pattern, match the PS pattern or match neither* 

The numbers of matches to the two predicted patterns for these 
two problem sets are presented in Table 3-5. These results may be 
sumriarized as follows; 

1* Approximately half of the subject may be described as following 
the exclusion strategy in both sets of problems. 

2. Approximately 10^ of the subjects appear to be following 
the preference strategy in both sets of problems. 

3. Approximately 23^ may be described by one strategy on one 
problem set and the other on the other problem set. 

4. Approximately 20^ match one of the models on one problem set 
and match neither model on the other. 



Table 3-5 



Number of Subjects with Response Patterns Matching Patterns 
Predicted by the Exclusion Strategy (E) and the Preference Strategy (P) 



Item set 3 Item set 4 No. of Subjects 



E E 19 

P P 4 

E P 3 

PE 6 

E 4 

P 2 

E - 0 

P - 2 



Total 



40 



One or the other of the models thus alone accounts for the response 
patterns of 23 of the 40 subjects. Let us consider those 9 subjects 
classified as E-P and P-E in Table 3-5. To account for their response 
patterns, we would need to assume that they switch strategies. Is such 
an assumption justified? It seems reasonable that some subjects might 
switch from the P-strategy to the E-strategy; if they stop to check 
their P-strategy response pattern against all of the information they 
would discover inconsistencies. For three of the six subjects, there is 
evidence from their response and information sheets that they do in fact 
switch from one strategy to another. Subject 32 made several errors in 
recording the exclusion information, but recorded the preference 
information correctly. This subject began by getting the first five 
items correct, which, given his recorded information, he could only do 
by the preference strategy. On the second set of five items, he gave 
the predicted PS pattern. On the third set of five items, the subject, 
apparently switching to the E strategy gave the set of incorrect responses 
that would be obtained by applying the E strategy to his recorded 
information. Finally, he missed all of the last five items, all of 
which he would have passed had he continued using the P-strategy. 

Subject 15 began the third set of items with the PS pattern, and had 
crossed these responses off. The correct answer had been supplied for the 
first item of the set, and the other items had been left blank. This 
suggests that the subject found a set of responses using the P strategy, 
noticed the contradictions, and had begun correcting the items when time 
was called. Subject 27 made notes on his response page for set 3 which 
suggested that he ’-rats attempting to use the E-strategy. The particular 
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cases suggest that in at least some of the instances, the subjects are 
accurately described by assuming that they switch strategies during the 
course of a test* 

Properties of Subject s Classified by ES and PS Concepts * 

It remains to investigate whether the concepts used in the 
classifications discussed above have any relationship to variables 
outside the specific test situation* Table 3-6 presents results 
comparing the various classifications of subjects on the syllogisms 
and word matrices tests* Since subjects using the E strategy rely 
heavily on both information organization and logical reasoning, we 
might expect them to excel the subjects using the P strategy with respect 
to these tests* The t-test comparing subjects who use the P-strategy 
at one time or another with subjects who never use the P-strategy 
bears out this expectation* (Table 3-7) 

Discussion 

This study was conducted in an attempt (a) to develop a measurement 
procedure to provide evidence of processes by means of which persons 
solve problems, and (b) to explore ways of describing individuals in 
terms of their problem solving processes* The measurement techniques 
employed consisted of a data gathering procedure which renders overt 
the information being used by the problem solver and some features of its 
organization, and the construction of problems in such a way that differing 
problem solving processes are revealed by predictable response patterns* 

The approach to desci-ibing individuals, is to construct models which 
employ hypothetical processes, and to match the performance of individuals 
with ^hat of alternative models* 
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cases suggest that in at least some of the instances, the subjects are 
accurately described by assuming that they switch strategies during the 
course of a test* 

Properties of Subjects Classified by ES and PS Concepts * 

It remains to investigate whether the concepts oised in the 
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and word matrices tests* Since subjects using the E strategy rely 
heavily on both information organization and logical reasoning, we 
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to these tests* The t-test comparing subjects who use the P-strategy 
at one time or another with subjects who never use the P-strategy 
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This study was conducted in an attempt (a) to develop a measurement 
procedure to provide evidence of processes by means of which persons 
solve problems, and (b) to explore ways of describing individuals in 
terms of their problem solving processes* The measurement techniques 
employed consisted of a data gathering procedure which renders overt 
the Information being used by the problem solver and some featiares of its 
organization, and the construction of problems in such a way that differing 
problem solving processes are revealed by predictable response patterns* 

The approach to desczdbing individuals, is to construct models which 
employ hypothetical processes, and to match the performance of Individuals 
with that of alternative models* 
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Table 3-6 



Mean Scores on Word Matrices and Syllogisms 



Classification of 



Subjects 


Word Matrices 


E-E 


73.4 


P-P 


60.3 


E-P 


54.0 


P-E 


66.7 


-E 


74.7 


-P or P- 


69.8 



Syllogisms 

20.3 

18.5 

16.3 

17.0 

19.5 

16.0 



Tabls 3-7 



Mean of Subjects on Word Matrices and Syllogisms 
Using and Not Using Preference Strategy (P) 



Word Matrices Syllogisms 



Mean of Subjects using P-strategy (N=17) 


63.6 


17.00 


Mean of Subjects not using P-strategy (N=23) 


73.0 


20.04 


Difference between means 


9.4 


3.04 




2.39 


1.17 


t 


2.54* 


2.60* 



* 

Significant at .05 level 
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The results of this study indicate that both the data gathering 
and description procedures are useful^ albeit primitive techniques. 

We suggest here some research efforts needed to develop the techniques. 
Most of the studies suggested call for more intensive information about 
the subjects* processes > using ”think-aloud” protocols, and for refining 
the instruments to be more sensitive to variations called for in model 
construction. 

We have been able to account for reponse patterns for most of our 
subjects by applying hypothesized strategies to discrete sets of items. 
However, it is clear that for many of our subjects, different models 
best represent their behavior on sets of problems solved at different 
times. If we are to account for the behavior of these subjects, we will 
need to define a mechanism governing the switching from one stragey to 
another. Is there a noticing or learning mechanism which leads subjects 
to switch from one strategy to another? If so, what signals its 
occurrence, and how can it be expressed in operational terms? We must 
consider models which employ both strategies, with rules to govern which 
is to be operating on a give problem at a given time. 

Other research, is needed to find whether the strategies which describe 
behavior in this particular type of problem can be genersQized to other 
problems, and whether we can predict from other data about individuals 
which particular model will be applicable to them. 

The procedures and point of view suggested here have long range 
implications for both theoretical and practical psychology. A series 
of studies using such an approach can illuminate a process theory of 
intelligence, and the procedures can provide dependent variables for 
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experimental stiidies of variables which affect process* The applied 
uses for such an approach include the diagnosis of difficulty in problem 
solving, the construction of remedial programs, and the assessment of 

e 

effects of educational treatment* 
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Appendix A 



Problem Analysis Exercise Used in Section 2 



In this exercise, you are presented with a problem and are asked to an- 
alyze the problem and propose solutions at three different stages. You will 
receive feedback after each stage, so that there will be increasing infor- 
mation available. The problem is a realistic educational one; there is no 
single ” correct?* solution . The object is to develop a reasonable plan for 
the solution based upon the available information and your understanding of 
students and teaching. 

Please keep this statement of the problem until you have completed all 
three stages . 

The Problem ; You have just taken a new teaching job in an urban high 

\ 

school. You have been assigned a sophomoie section of a course in your sub- 
ject matter. The course you are teaching is part of a new curriculum recent- 
ly developed by teachers in your area in cooperation with a local university. 
It is designed to encourage critical thinking about the subject matter, and 
the 4-year sequence is planned to provide .a cumulative educational experi- 
ence. 

Two days before the start of class, Mrs. Reynolds, a teacher in your 
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department, stopped to talk to you. ”I notice that you will have Arthur 
Green in your class,” she said. ”I would appreciate it if you could give 
him some special attention. He came into my Freshman class late last se- 
mester. He seems to be a bright boy, but had a great deal of difficulty 
catching on to the material. I passed him, thinking that I would be able 
to work with him more this year, but it turns out that he will be in your 
section, rather than mine.” 



In response to your request for more information, Mrs. Reynolds said: 
’’Well, on exams he makes very little effort at analyzing a problem. He 
usually tries a question, but most often, he just writes down a definition 
or two, or quotes a passage from the book. He doesn't even try to show re- 
lationships between what he has written and the problem posed by the ques- 
tion. He never participates voluntarily in class and when I call on him, 
he blushes and stammers so that the whole class is uncomfortable for him.” 

Instructions . You are asked to comment on this problem at three 
stages . Stage 1_ occurs now, before you have met Arthur, and as you develop 
a preliminary plan of attack. Stage II occurs just after the opening of 
class. You have obtained some additional information, and have arranged 
for an interview with Arthur. You may ask for appropriate information at 
each stage. At Stage I, you may seek information from Mrs. Reynolds, from 
other teachers, or from school records. At Stages II and III you may re- 
quest information from these sources and from Arthur himself as well. 



> 

j 

/ 
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Appendix A 
(Continued) 



Name 

Your Subject 

Stage 

Date 



Problem Analysis 

1. What is your present analysis of the problem (e.g. hypotheses, expec*- 
tations, suspected determinants of the problem)? 



2. List items of additional information you would like to have. 
(Please number the requests.) 

Item 



Source 









Appendix A 
(Continued) 



What is your present plan for attacking the problem? 





Appendix B 

Scoring Guide for Problem Analysis Exercise 
ANALYSIS (Question 1) 

Classified as M (multiple) or S (single), depending upon whether the 
subject considers multiple alternatives concerning the bases of the problem 
or zeroes in on a single hypothesis. M or S is characterized less by the 
number (one as opposed to more than one) of causes offered than the nature 
of the subject’s attitude toward the causes. If his attitude is flexible, 
his response is coded as M. If his attitude is fixed, not subject to change, 
his response is coded as S. A multiple response states explicitly that the 
problem could be caused by any one of a number of factors, or a. combination 
of several of them. Frequently, the subject itemizes several possible causes; 
sometimes he doesn’t list them but simply says there are many possibilities. 

The distinguishing feature of a multiple response is a tentative attitude, 
indicated by such words as "perhaps,” "maybe," "possibly," and such phrases 
as "This problem might be..," "There are many (or a variety of) possible 
causes." A single response, on the other hand, is characterized by rigidity. 
The subject usually gives only one cause. If he lists more than one, he does 
so in an assertive manner, as if to say, "I already have the problem completely 
analyzed. There are no other possibilities than the ones I have given here." 
INFORMATION CQuestion 2) 

Step 1. Count the number of items requested. If the subject violates 
the rules of the experiment (for example, if he asks a question directly of 
Arthur in Stage I), that question is not counted. If the subject asks the 
same question more than once within a given stage (just uses different words 
to ask the same thing), he is only given credit for the question the first 
time it appears. 




t 
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Step 2. Compute the average number of items asked within a given stage. 
Aj*3, A2«4, A^=2. Trichotomize the data: 

H = above average 
A = at the average 
L = below average. 

Step 3. The object here is to dichotomize the data from Step 2. 
Categories are H, above the mean; L, below the mean. For Stage III, all A»s 
become H»s. This produces an almost equal number of L»s and H»s, the median 
falling between 2 and 3. For Stages I and II, look at all the A»s. If the 
items come mostly from one source (all from school records, or all from 
Arthur, for exanqple), classify as L. If the items are mostly an elaboration 
of a single idea (several questions about his family situation, all closely 
related, for example), classify as L. Otherwise, classify as H. 

STRATEGY (Question 3) 

Step 1. Classify as U (unilateral), 0 (open), C (conditional). 

A unilateral response is characterized by rigidity. The subject has the 
attitude, *'I will do this regardless of what new information reveals, re- 
gardless of whether or not my method appears to work.” Usually, only one 
idea is offered. If more than one is given, they are assumed to be carried 
out simultaneously, rather than being considered as several possible alterna- 
tives. An open response admits of the possibility of change. The subject 
realizes that additional information may indicate a change in plans, but he 
doesn't specify the precise relationship between information asked in the 
information category and his strategy. If he suggests several strategies, 
he does so with the idea that he will use one or several or none, depending 
on what happens in the future. If he only gives one plan of action, he 
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states that others are possible. He says, might do this,** or **I could do 
that,** whereas a unilateral subject says, **I will do this.** A subject with 
an open response may propose no method at all, saying he wants to learn more 
before making any decisions. 

A condition response is like an open response except that the former 
draws an explicit relation between information sought and the plan of action 
he will eventually adopt. It is an explicit branching strategy, the choice 
of alternatives dependent on definite criteria. The response is almost always 
stated in an if-then structure. 

Step 2. Combine all C responses with 0*s. 



Appendix C 

Problems Used in Section 3 
Couples Test 

This test contains several puzzles based on the romantic adventures 
of five young ladies: Ann, Betty, Cathy, Doris, and Ellen. Each of the 

girls has definite ideas about acceptable boyfriends. They accept dates 
according to the rules given on the cards that you have. 

Take a few memonts to read over the cards. 

Now make a note of the information given on the cards on the Infor- 
mation Sheet that you have. You will use this sheet to refer to while solv- 
ing the puzzles. You may write down the information in any order you wish. 

When you have finished, return the cards to the experimenter. 

Remember that each girl obeys her own rules consistently in the pro- 
blems that follow. 

Do the problems ”in your head”. Do not make notes or write down anything 
except the answers you wish to give. 



PLEASE DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 
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Appendix C 
(Continued) 



The five girls have recently been escorted by five young men: Al, Bert 
Carl, Don, and Ed. One of the men is an architect, one a bartender, 
one a chemist, one a dentist, and one an engineer. Your problem is to 

discover which is which. 

Ann has dated Al. 

Ellen has dated Bert. 

Doris has dated Don. 

Betty had dated Al. 

Cathy has dated Ed. 

Betty had dated Carl. 

Write the profession of each man beside his name below. 

name profession 

Al 

Bert — 

Carl — 

Don — 

Ed — 



PLEASE DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 



Appendix C 
(Continued) 

2. Time has passed and, being girls, our girls have changed their boyfriends, 
but not their occupational prejudices. Their new circle includes Fred, 
George, Hal, Irv, and Jack, one of whom is an architect, one a bartender, 
one a chemist, one a dentist, and one an engineer. 

Ellen has dated George. 

Betty has dated Fred. 

Doris has dated Irv. 

Ann has dated Jack. 

Cathy has dated Fred. 

Betty has dated Hal. 

Write the profession of each man beside his name below. 

name profession 

Fred 

George 

Hal 

Irv 

Jack 

PLEASE DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 
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Appendix C 
(Continued) 

3. Again, the men have changed, but the rules remain the same. The men 
are Ken, Les, Mel, Ned, and Othello. One is an architect, one a bar 
tender, one a chemist, one a dentist and one an engineer. 

Cathy has dated Les. 

Ann has dated Ken. 

Doris has dated Ned. 

Ellen has dated Les. 

Betty has dated Othello. 

Doris has dated Ken. 

Write the profession of each man beside his name below. 

name profession 

Ken 

Les 

Mel — - 

Ned 

Othello _ 



PLEASE DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO 
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Again, the rules remain the same. The men are Paul, Quentin, Roger, 
Sam, and Tom. One is an architect, one a bartender, one a chemist, 

one a dentist and one an engineer. 

Doris has dated Sam. 



Ellen has dated Quentin. 
Ann has dated Paul. 



Cathy has dated Tom. 
Ellen has dated Paul. 



Betty has dated Roger. 



Write the profession of each man beside his naikie below. 

name profession 



Paul 

Quentin 

Roger 

Sam 

Tom 
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Appendix D 

Rules for Preference and Exclusion Strategies 
Rules for Preference Strategy 

1. a) List problem designations, P:P^-P^ . 
b) Set up a solution list, S:Sj-Sg 

[Note: an entry in S^ may be one or two components.] 

2. Select initial problem to be solved, P^ • 

3. Solve problem P^ . 

a) Look up pairs involving P^^ . 

b) List preferences associated with pairs involving P^ in L . 

c) How many entries in L ? 
c-0. If 0 , go to d . 

c>l. If 1 , is it the same as a previous solution (i.e., an 
entry in S preceding ^ 

c-11. If yes, do 5 . 
c-12. If no, enter entry in L as S^^ . 
c-2. If 2 , 

c-21. Is the first the same as a previous solution? 

c-211. If yes, do 5 . 

c-212. If no, add entry in L to • . 

c-22. Is the second the same as a previous solution? 

c-21. If yes, do 5 . 

c-22. If no, add entry in L to S. . 

n 
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Appendix D, Continued. 

d) Is problem terminal for set? 
d-1. If yes, go to 4 . 

d-2. If no, substitute + 1 for ; go to 3-a . 

4. Are any of blank? 

(i.e., are there any unanswered questions in set?} 

a) If yes, 

a-1. Designate blank solution as . 
a- 2. How many solutions (Professions) hi^e not occurred? 
a-21. If 1 , record that solution as S^, . 
a-22. If more than one, leave blank. 

b) If no, go to next set. 

5. Resolve a conflict between solutions. 

a) Designate the conflicting entries in S as and . 

b) Is entry in S forbidden by exclusion rules? 
b-1. If yes, erase entry in S . 

b-2. If no, repeat 5-b for . 

c) Does conflict still exist: 

c-1. If yes, leave S and S blank. 

W 

c-2. If no, return to 3-c . 
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Appendix D, Continued. 

Rules for Exclusion Strategy 
1. a) List problem designation, P:Pj-Pg . 

b) Set up a solution list, S:S^-S^ . 

Select initial problem to be solved, P^ . 

3. Solve problem P^ . 

a) Look up pairs involving P^ . 

b) List solutions excluded by pairs. 

c) List non-excluded solutions in L. 

d) How many entries in L ? 
d-0. If 0, go to c . 

d-1. If 1 , enter entry in Lj as S^ , and go to e . 

d>2. If more than one, designate entries L^, L^, ; 

do d-21 for all entries in L . 

d-21. Is L^ the same as a previous solution? 

d-211. If yes, erase L^ from L, return to d . 
d-212. If no, move L^ to end of list L . 

Substitute L2 for L^ ; return to d . 

e) Is problem terminal for set: 

e-1. If yes, go to 4 . 

e-2. If no, substitute P +1 for P ; 

n n 

go to 5 . 

4. Are any of S^-S^ blank? 

a) If yes. 
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Continued 



a-1. Designate blank solution as . 
a- 2. How many solutions have not occurred? 

a-21. If 1 , record that solution as S ; 

y 

go to next set. 

a-22. If more than one, leave S blank. 

y 

b) If no, go to next set. 
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Appendix D, Continued. 



Example of Application of the Preference Strategy (Problem Set 1) 

Problem 

Being 

Solved Application of Instruction Comment 



1-a. Pj=A, P 2 *B, Pj=C, P^=D, Pg=E . 




P 




Pj-B 



2. P,-A . 

3-a. Akf BA . 

-b. Lj* B, E . 

-c. 2 . 

c-21. No; S^sB . 

c-22. No; Sj«E . 

•d. No. Solve problem P^ . 

3>a. EB . 

-b. Lj=A . 

-c. 1. No. 

c**12. • 

-d. No. Solve Problem P^. 






3-a. BC 

-b. Lj»B 
-c. 1 



(Present state of S is: 

^^BjE; S2~Aj Sg* ^ ^ . ) 



(Present state of S is: 

S^^BjE; S2~Ay ) 
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to next set. 




Appendix D, Continued. 

Pv.mpi. nf Anplication of Exclusion Strategy, CProblem Set 1) 



Problem 

Being 

Solved 



Application of Instruction 



Comment 



1-a. P =A, P 2 *®> ^ 5 ”^ * 

-b. Sj=_, S 2 =_, Sj=_, S^=_, Sg=_ . 

2. Pj*A . 




Pj?»A 



P2=B 



3-a. AA, BA . 

-b. A, B, C, D . 
— c. L*E • 

-d. 1 . 



d*^!. . 

-e. No. Solve Problem ?2 • 



(Present State of S is: 
Si=E, S2*_, S^=_, Sg=_.) 



3-a. EB . 

-b . B j C j D . 



-c. L“Aj E . 

-d. 2 . 

d- 2 . Lj=A, * 

d-21. No. 

ll- ®E f 






P4-D 
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-e. No. Solve Problem P^ . 



S-a. BC . 

-b. C, D . 

-c. L=A, 6, E . 
-d. 3 . 



d-2. Lj=A, ^2^6, • 



-d. 2 . 



d*"2. L2~B . 



d-21. No. 



-d. 2 . 



d**2. L2*B . 



d-21. Yes; L=B . 



-d. 1 . 



d-1. 83=8 . 

•e. No. Solve P^ . 



3-a. DD . 

-b. D, E . 

-c. L«A, B, C . 

-d. 3 . 



(Present state of S is: 
SjSsE, S2®A, ^5“—*^ 



d-21. Yes; L=B, E . 



d— 2. L^— Ay L2— By . 

d-21. Yes; L=By C . 



(Present state of S is: 
Si=Ey S2«Ay S^=B, ^ 5 “—*^ 
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Continued 



3-d. 2 . 

d— 2. Lj— Bj . 

d-21. Yes; L=C . 

-d. 1 . 

d-1. S^=C . 

-e. No. Solve Problem Pg . (Present state of S is: 

S^eE, S2*Aj Sg«B, S^=C, Sg*^.) 



-b. B, E . 

-c. A, B, D . 

-d. 3 . 

d— 2. L2 *Bj . 

di-21. Yes; L»B, D . 

-d. 2 . 

d**2. Lj®B, ^2^^ * 

d-21. Yes; L«D . 

^d. 1 . 

d-1, ^5°^ * 

-e. Yes. 

4. No; go to next set. 
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